Guideline: Using TOGAF to Define and Govern SOAs

Overview

This chapter discusses:

Service Oriented Architecture (SOA) as an architectural style
Factors relating to the adoption and deployment of SOA within the enterprise
Correspondences between SOA and TOGAF terminology
The definition and structure of service contracts

Note:: The Open Group SOA Working Group is currently working to develop a Practical Guide that will enable a certified TOGAF practitioner to use TOGAF to develop an SOA. More information on the SOA Working Group can be found at www.opengroup.org/projects/soa. More information on the SOA/TOGAF Practical Guide project can be found at www.opengroup.org/projects/soa-togaf.

Introduction

As the business environment becomes more sophisticated, the challenges facing large organizations are shifting away from questions of efficiency and automation towards questions of complexity management and business agility. Complex webs of existing applications and interfaces create highly complex IT landscapes where change becomes more and more difficult and the impacts of change become harder to predict and understand.

The concept of an SOA provides an architectural style that is specifically intended to simplify the business and the interoperation of different parts of that business. By structuring capability as meaningful, granular services as opposed to opaque, siloed business units, it becomes possible to quickly identify functional capabilities of an organization and to avoid duplicating similar capabilities across different areas of the organization. By standardizing the behavior and interoperation of services, it is possible to limit the impacts of change and also to understand in advance the likely chain of impacts.

From a software development perspective, SOA focuses on structuring applications in a way that facilitates system flexibility and agility - a necessity in today's complex and fast-moving business environment. SOA aims to break down traditional application silos into portfolios of more granular services that operate in open and interoperable ways, whilst extracting commodity capability into a virtualized infrastructure platform of shared re-usable utility services.

Business-Led SOA Community

In response to the business opportunities presented by SOA technology, a growing community of professionals is examining SOA as a business opportunity and looking at how business can be restructured to gain from more flexible, agile, and open IT solutions. Rather than addressing challenges of software engineering, the business-led SOA community focuses on issues such as sharing of business capability, sourcing of business capability, and exposure of business capabilities to new audiences and channels.

Fundamental to the business-led SOA approach is:

Rich domain knowledge of both horizontal, cross-cutting concerns, such as human resources (HR), finance, and procurement alongside vertical, industry-specific concerns
A structured, quantitative understanding of business value, costs, differentiators, and quality measures
Broad understanding of current capability, showing both business capability and how it is supported by IT
Broad understanding of the feasibility and viability of particular SOA technology-driven solutions

This remit is in sharp contrast to the technology-centric, "developer-led" SOA community that maintains a core focus on the similarities across industries, organizations, and products to achieve benefit.

Business- & Developer-Led SOA Communities

Although the business-led and developer-led SOA communities maintain a different focus, their activities are complementary and intersect at the concept of a "service" (see Business-Led versus Developer-Led SOA Communities).

Figure: Business-Led versus Developer-Led SOA Communities

Business-led SOA considers a business service to be a unit of business capability supported by a combination of people, process, and technology. A business service may be:

Fulfilled by manual processes, or may be fully automated
Fulfilled within an organization, or outsourced to a partner
Exposed to any combination of employees, customers, partners, and suppliers
Fulfilled at the point of use, at a divisional level, or as a corporate competency center

Defining business services allows an organization to differentiate between business capability, the fulfilment of business capability, and the consumption of business capability. This differentiation allows the organization to consider sourcing, procurement, federation, centralization, and channel exposure in a much more flexible way, prioritizing investment and focus in areas that are core differentiators, whilst cost controlling and divesting areas that are considered to be context to the business.

Developer-led SOA considers an information system service to be a unit of application code providing an open interface that is abstracted from its implementation. Information system services support separation of concerns between:

Encapsulation of business flow and application composition as Process Services
Encapsulation of application function as Application Services
Management of data access and persistence within Data Services
Commoditization and sharing of common utility functions, such as monitoring and security within Infrastructure Services

Creation of this separation of concerns between information system service types supports more effective application re-use and the creation of multiple composite applications from a single service portfolio. For example, all applications can share common security services, many application functions can share the same data services, and many process-centric applications can share the same application services.

Additionally, the separation of service types supports specialization of infrastructure and tooling to optimize development, maintenance, and operational performance. For example, business process services can be developed, viewed, and maintained in a Business Process Management (BPM) environment, specifically designed for flow definition.

Finally, as all information system services operate in a consistent manner, a common SOA platform or SOA fabric can be deployed that manages hygiene factors for services in a consistent way. For example, service communication can be standardized and virtualized through the use of Enterprise Service Bus (ESB) infrastructure. Application monitoring can be abstracted from particular implementation stacks using agents and devices that monitor standards-based communications.

Complexities Arising from SOA

Alignment between business and IT within an organization is a fundamental challenge facing SOA adopters. However, even with a fully aligned organization, there are still significant differences between an SOA landscape and a traditional IT landscape that create new points of stress and focus.

In the past the "application" concept has provided the key to understanding the link between business and IT. Applications historically map closely onto organizational silos and as the number of applications is small, it is a straightforward task to govern these applications. However, within an SOA the concept of the application is augmented by the concept finer-grained application services, creating new challenges and complexity.

Whilst providing much greater business flexibility and agility, the breaking up of siloed business functions and applications into services comes at a cost, in that it creates a much more fine-grained IT landscape that needs to be managed. As a by-product or producing finer-grained capabilities, an increased volume of services must be managed (100s or 1000s of services as opposed to 10s or 100s of applications) with correspondingly increased complexity around the usage of and interaction between services:

New stress points are created around understanding the relationships between technology portfolio and service portfolio
New stress points are created around SLA definition, governance, and impact management
New stress points are created around tracing business to IT
New stress points are created around communication, alignment, and semantics
New stress points are created around platform and interoperability
New stress points are created around performance visibility and optimization

Technology can provide tooling to address many of these stress points, but the real issue here is that effective operation of an SOA requires a much more formalized understanding of the IT landscape with explicit linkages to the business it supports. Without this understanding, there is a very real risk that the possibilities of SOA will lead to an organically developed IT landscape, characterized by:

Proliferation of unplanned, misaligned services at inappropriate levels of granularity, known as "service sprawl"
Inability to carry out impact assessment and consequent overspend on infrastructure or poor quality of service
Multiple technology stacks that are costly to support and do not interoperate, creating islands of services tied to implementation specifics that result in a brittle IT landscape and high operational costs, due to more complex service management and IT operations
Inability for potential service consumers to identify services for re-use, resulting in duplicated capability, lack of visibility, and increased integration complexity

How Enterprise Architecture Supports SOA

Enterprise architecture is the application of architectural discipline to the end-to-end enterprise, treating the enterprise or industry value-chain as a system. What enterprise architecture provides in an SOA context is a set of tools and techniques to link top-down business-led SOA to bottom-up developer-led SOA in a robust and maintainable way that addresses many of the non-technical challenges associated with SOA adoption.

Enterprise architecture discipline provides the following tools and techniques to assist organizations with SOAs:

Enterprise architecture defines structured traceable representations of business and technology that link IT assets to the business they support in a clear and measurable way. These models in turn support impact assessment and portfolio management against a much richer context.
Enterprise architecture defines principles, constraints, frameworks, patterns, and standards that form the basis of design governance, ensuring aligned services, interoperability, and re-use.
Enterprise architecture links many different perspectives to a single business problem (e.g., business, data, application, technology, abstracted, concrete, etc.) providing a consistent model to address various problem domains and extensive test for completeness.
Enterprise architecture provides consistent abstractions of high-level strategies and project deliverables, enabling both bottom-up and top-down outputs to be collated in a shared repository to support planning and analysis.

Using these techniques, enterprise architecture becomes a foundation for service-orienting an organization, because:

It links SOA stakeholders together, ensuring that the needs of each stakeholder community are met and that each stakeholder community is aware of appropriate context.
It provides a link from business to IT that can be used to justify the cost of IT re-engineering against business value.
It shows which services should be built and how they should be re-used.
It shows how services should be designed and how platforms must interoperate.
It provides a repository to hold and maintain design-related information on an ongoing basis.

SOA and TOGAF

A number of concepts are captured in the TOGAF content metamodel that support the modeling of SOA concepts:

Function: A function is a thing that a business does. Services support functions, are functions, and have functions, but functions are not necessarily services. Services have more specific constraints than functions.
Business Service: A business service is a thing that a business does that has a defined, measured interface and has contracts with consumers of the service. A business service is supported by combinations of people, process, and technology.
Information System Service: An information system service is a thing that a business does that has a defined, measured interface and has contracts with consumers of the service. Information system services are directly supported by applications and have associations to SOA service interfaces.
Application Component: An application component is a configured and deployed system, or independently governed piece of a configured and deployed system. Application components provide information system services. Application components can be physical applications and also logical applications that group together applications of a similar type.
Technology Component: A technology component is a piece of software or hardware that can be purchased from an internal or external supplier. Technology components are configured, combined, built on, and deployed in order to produce application components.

TOGAF Concepts Mapped to SOA Terminology shows how these TOGAF concepts (in blue) can be equated to common SOA terminology (in red).

Figure: TOGAF Concepts Mapped to SOA Terminology

Within the TOGAF Architecture Development Method (ADM), specific consideration is made to SOA concerns at various stages to ensure that appropriate pre-requisites are in place.

TOGAF Phase	SOA Concept	Benefits to an SOA Initiative
Preliminary	The TOGAF framework provides a metamodel and process that is capable of incorporating both business-led and developer-led SOA concepts in a holistic framework.	Using TOGAF will provide a direct linkage between business-led and developer-led communities, initiatives and benefits within an organization.
Architecture Vision	The TOGAF ADM includes specific steps to address SOA concerns, including: Consideration of business capability - which capabilities are most valuable, volatile, and differentiating for the business? Consideration of technology capability - how technically mature is the organization and how mature does it desire to be? Consideration of IT governance impacts - what impacts is the Architecture Vision going to have on current IT governance models?	TOGAF Business Capability Assessments provide an opportunity to look at the business services that exist or are desired and how these business services can be realized. This Business Capability Assessment formalizes the work of business-led SOA stakeholders in a way that can be transferred to developer-led initiatives. The TOGAF Technology Capability Assessment provides an opportunity to look at technology risk and maturity, allowing the organization to make informed decisions on how fast to adopt SOA technology and also to select strategies for deployment of SOA and service technology platforms. The TOGAF IT governance assessment provides an opportunity to identify SOA-related governance impacts and to factor any new capability requirements into the overall Target Architecture.
Business Architecture	The Business Architecture phase elaborates the capabilities of the business and defines explicit portfolios of business services, accompanied by service contracts that formalize service consumption.	Business services are explicitly identified and tied to ownership, usage, and business value, forming the detail of a business-led SOA strategy in a way that can be linked into a developer-led world.
Information Systems Architectures	The Information Systems Architectures phase shows how the identified business services map to applications and data.	Information system services are explicitly identified and tied to business service, data encapsulation, and applications, elaborating a high-level framework for developer-led SOA implementation.
Technology Architecture	The Technology Architecture phase shows how application capabilities identified in the Information Systems Architecture are mapped onto SOA platforms and off-the-shelf SOA services, providing a blueprint for implementation.	SOA technology selection is not carried out in isolation as a pure feature comparison between products. SOA technology architectures are developed with traceable reference to: Business ownership and value A defined organizational position on baseline and target technology maturity Service contracts that identify service consumers, SLAs, and non-functional requirements for services Landscape visibility of how technologies will support delivery of applications and information system services Consideration of the IT governance model, requirements, and issues
Opportunities & Solutions Migration Planning	The TOGAF ADM allows for identification of improvement opportunities and then selection, prioritization, and sequencing of those opportunities according to architectural analysis and stakeholder need.	Development of a multi-disciplinary Architecture Roadmap allows SOA capability to be incrementally developed, including proof-of-concept, pilot, and mainstream SOA enabling. Considerations - such as standards, guidelines, technology selection, design governance, operational governance, and security - can be considered holistically and charted for an organization in a way that supports incremental capability development, but avoids organic growth of "accidental architectures".
Implementation Governance	TOGAF provides specific processes for design governance during the implementation of an architecture.	TOGAF identifies the need for design governance, which can include SOA design governance. This approach provides a framework in which to apply an organization's standards, guidelines, and specifications to implementation projects.
Architecture Change Management	TOGAF allows for implementation issues and external factors to be incorporated into the architecture, allowing the overall approach to be refined.	Lessons learned from proof-of-concept and pilot activities can be leveraged and used to shape the strategy from a bottom-up perspective.

Guidelines for Service Contract Definition

Service Qualities and TOGAF

Besides the set of components making up the Technical Reference Model (TRM) - see Part VI, Foundation Architecture: Technical Reference Model , there is a set of attributes or "qualities" that are applicable across the components.

The Detailed TRM diagram captures this concept by depicting the TRM components sitting on a backplane of Qualities.

Qualities are specified in detail during the development of the Target Architecture, and then used on an ongoing basis to govern and measure the success of the architecture.

Service contracts are an example of such qualities. This section defines further detail on how to create appropriate contracts for architectural services.

Purpose of a Service Contract

It is commonly understood that specifying the external characteristics of required IT capabilities, in a way that aligns with business objectives, supports creation of higher quality architecture.

The design and parameters around each service are analyzed more thoroughly, re-use potential is maximized, and governance parameters are aligned with business expectation and focus.

As Business Architecture is all about defining a formal corporate business model, prior to modeling service boundaries, it is this phase in an enterprise architecture engagement where "service modeling" should be considered. The approach must center around logical building blocks representing business services, and concentrate on defining the interfaces and Service Level Agreements (SLAs) between service providers and service consumers.

For services to interact, they do not need to share anything but a formal agreement that describes each service and defines the terms of the information exchange between consumer and supplier. A contract is a set of metadata that describes how parties will interact both functionally and non-functionally.

The outcome of this approach is that IT systems can respond to changes in policies and contracts without the need for more tightly coupled, inflexible programming - essentially, the policy needs to be changed and the contract adjusted, and the consumers and providers simply follow the amended contract. This is the ultimate vision of SOA.

The link between a business service and its service contracts is important - changes in an organization's business strategy and vision will trigger changes in these contracts - and their impact on the enterprise architecture needs to be understood.

Service Governance Considerations

An enterprise architecture includes many independent and self-contained moving parts - components which are re-used widely across the organization that become a vital part of mission-critical business processes.

If this is the case:

What happens when a service is changed?
Who owns the service?
What SLAs govern the service?
Who needs to test the service before it goes into production?
How can you be sure the service you are consuming is of high quality?
How can you be sure a new service is compliant with IT, business, and regulatory policies?
How can you ensure predictable uptime of a service?

These questions illustrate the need for service governance. Service governance is about managing the quality, consistency, predictability, change, and inter-dependencies of services.

Service governance is a topic in itself. The IT Infrastructure Library (ITIL) Service Delivery Guide provides detailed guidance on the definition and management of SLAs and service governance.

A significant challenge facing IT organizations today is that while the management of service quality is paramount, simply having quality is not enough.

For the first time, quality must be proven and demonstrable to consumers to gain their trust and create an effective shared-service environment.

The cost of an ungoverned enterprise architecture is a lack of re-use, disruption and failure of business process, escalation of support costs resulting from service outages, security breaches, and non-compliance with enterprise or governmental regulations.

Service governance requires a cohesive strategy involving multiple elements that include service contracts, service policies, service lifecycle management, and service metadata.

Service Contracts

Contracts are key architectural tools for communicating and enforcing policies, as well as other requirements in a heterogeneous and distributed IT environment. Just as a business contract ensures a healthy commercial relationship, a service contract ensures a healthy provider/consumer relationship, and helps to establish an agreement and maintain trust between these parties. In other words, a service contract should provide a precise and unambiguous agreement for how the provider and consumer interact. Contracts are typically unique to a specific provider/consumer relationship, and they act as the container for both formal policies, as well as agreements that are unique to the parties.

Service Policies

The nature of SOA (highly distributed, heterogeneous, and very dynamic) means that it is critical for SOA artifacts to be governed by specific business, technical, and regulatory policies. In SOA, policies are not hard coded into a specific application, but are coupled to services.

An SOA policy defines configurable rules and conditions that affect services during both design time and run time. This means that policies must be used to validate services before they are published, and as a basis for enforcing specific standards and behaviors at run time.

Non-SOA applications also require effective definition and management of interaction policies. However, due to the coarse-grained, siloed nature of traditional systems, policy is less significant and is typically encoded into the application code and platform rather than being governed at the architectural and design level.

Service Lifecycle Management

Service artifacts need to be managed across a complete lifecycle.

Service lifecycle management is about:

Ensuring the quality, performance, and applicability of services that are published
Providing a means for consumers to discover and re-use services and other artifacts
Managing versions, security, and state change of services and other artifacts
Assessing and managing the impact of change across a network of consumers
The lifecycle of individual services as they are designed, built, and deployed (which is primarily the concern of the service provider)
The lifecycle of a network of services (in which services are accessed and used by changing populations of service consumers, and where the lifecycle primarily concerns those consumers)

Service Metadata

In traditional IT environments, metadata is typically defined within the code of systems and applications.

Externalizing service metadata from the native system enables the classification and governance of these independent services. Thus, metadata becomes a key artifact that needs to be managed within the overall architecture.

Content and Structure of a Service Contract

In any service consumer/provider relationship, there are two different senses of contract:

A Governance Contract between two business entities which specifies what interaction is needed, inputs, outputs, SLAs, OLAs, and KPIs
A Operational Contract which specifies the actual communication protocols and message formats

In TOGAF, at the architecture level, the main concern is the Governance Contract.

Just as legal contracts are documents that outline a set of enforceable rules and agreements written in a language that both parties can understand, in IT a contract should be defined that stipulates IT "rules of engagement" in a standardized manner that both the consumer and provider can understand.

It is during Business Architecture that definition of service contracts (in business terms) begins, before making decisions about how these services are to be implemented technically.

The service contract specifies the quality and integrity of the interaction between services. This specification allows the development of service-level objectives (irrespective of whether they are formalized into an SLA).

Service contracts form an important insight to developing the critical operational path, and they set the quality and security benchmarks for Application and Technology Architecture components.

If technically automated as an XML web service, a service contract can be comprised of a collection of service descriptions and documents including:

WSDL Definition: An XML-based service description that describes the public interface, protocol bindings, and message formats required to interact with a web service.
XSD Schema: An XSD defines a type of XML document in terms of constraints upon what elements and attributes may appear, their relationship to each other, and what types of data may be in them.
Policy.

However, there a range of technical interface methods available to exchange information, not just XML web services; for example,

IBM Websphere MQ
CORBA
Java Message Service (JMS)
Database Replication or Synchronization
Sending an ASCII file or email
Putting a paper form in an envelope and posting it
Simply displaying a sign on a wall for people to read

The contract is between whoever is providing the service, and whoever is consuming the service. Put it in the simplest terms, the contract provides details about the service being created by the provider. By agreeing to a contract, both parties understand exactly what will be provided, often before any decisions are made on the approach to realize a service (which may include outsourcing, purchasing a package, writing code, manual fulfilment, etc.).

This higher-level contract is crucially important, because these contracts frequently have not only technical implications but also business implications. A contract may include details of how the service will be authenticated, and so have details about authentication, encryption, and authorization. It may also include SLAs and how billing, metering, and monitoring will be done.

Contracts allow a provider company to create a single service, and then sell that service to many different customers, by simply creating a separate contract for each customer. A provider company may provide a service, but may charge more for that service if it responds more quickly to an identity, or provides a higher level of reliability. By including the information in the contract rather than the service, the service only needs to be created once. To re-use it with a different partner, only the contract needs to be changed.

Loose coupling mandates between parties should be able to conduct service fulfilment and optimize service realization as long as the terms of the agreed contract are met. Loose coupling is realized by implementing contracted interfaces on systems and making sure that those contracted relationships are enforced while allowing each party to change how they implement the contract independently.

Service Contract Template

Service contracts should be defined very carefully and clearly. The architecture performance will be based on these contract characteristics.

The contract should describe functional requirements; that is, what a provider will give to any consumer that chooses to abide by the terms of the contract. The contract should define what functionality is provided, what data it will return, or typically some combination of both.

Contracts must also specify non-functional requirements that detail not what the service does, but the way in which it goes about its business. This includes information both about the responsibility of the providers for providing their functionality and/or data as well as the expected responsibilities of the consumers of that information and what they will need to provide in return, such as availability, security, and other quality of service considerations.

A contract is an expression of the visible aspects of service behavior, and so contracts never include the data that providers and consumers actually exchange, or any specifics about how a provider or a consumer will meet the requirements of the contract. In addition, since consumers vary just as much as providers, there might be multiple contracts for a single service.

Attribute Type	Attribute	Description
General	Reference	A unique identifier within the architecture information for cross-reference, clarity, and differentiation from other similar artifacts.
General	Name	A suitable, preferably unique, short form name for the artifact.
General	Description	Name of the service. Should indicate in general terms what it does, but not be the only definition. A narrative of what the artifact is, what it does, and its relevance to the architecture.
General	Source	The source of the artifact, which may be a person or a document, along with a date to support traceability of the architecture.
General	Owner	The owner of the artifact is the name (person or group) who validated the details of this artifact; the person/team in charge of the service.
General	Type	The type of the service to help distinguish it in the layer in which it resides; e.g., data, process, functionality, presentation, functional.
General	Version	The version of the service contract.
Business	RACI	Responsible: The role is the person/team responsible for the deliverables of this contract/service. Accountable: Ultimate decision-maker in terms of this contract/service. Consulted: Who must be consulted before action is taken on this contract/service. This is two-way communication. These people have an impact on the decision and/or the execution of that decision. Informed: Who must be informed that a decision or action is being taken. This is a one-way communication. These people are impacted by the decision or execution of that decision, but have no control over the action.
Business	Service name "caller"	Name of the consuming service.
Business	Service name "called"	Name of the provider service.
Business	Functional Requirements	The functionality in specific bulleted items of what exactly this service accomplishes. The language should be such that it allows test cases to prove the functionality is accomplished.
Business	Importance to the process	What happens if the service is unavailable.
Business	Quality of information required	The quality that is expected from the service consumer in terms of input, and what quality is expected from the service provider in terms of output.
Business	Contract control requirements	How the contract will be monitored and controlled.
Business	Result control requirements	How the results of the service will be monitored and controlled.
Business	Quality of service	Determines the allowable failure rate.
Business	Service Level Agreement	Determines the amount of latency the service is allowed to have to perform its actions.
Non-functional Requirements	Throughput	Volume of transactions estimated; e.g., 100,000.
Non-functional Requirements	Throughput period	The period in which those transactions are expected, e.g., 100,000 per day.
Non-functional Requirements	Growth	The growth rate of transactions over time (estimated based on service take-up and business growth); e.g., 10,000.
Non-functional Requirements	Growth period	The period in which the growth is estimated to occur; e.g., 10,000 per year.
Non-functional Requirements	Service times	The available hours/days the service is needed; for example, 9 to 5 Monday to Friday (excluding Bank Holidays).
Non-functional Requirements	Peak profile short term	The profile of the short-term level of peak transactions; for example, 50% increase between hours of 10 to 12 am.
Non-functional Requirements	Peak profile long term	The profile of the long-term level of peak transactions; for example, 50% increase at month end.
Non-functional Requirements	Security requirements	Who can execute this service in terms of roles or individual partners, etc. and which invocation mechanism they can invoke.
Non-functional Requirements	Response requirements	The level and type of response required.
Technical	Invocation	The invocation means of the service. This includes the URL, interface, etc. There may be multiple invocation paths for the same service. There may be the same functionality for an internal and an external client, each with a different invocation means and interface. For example, SalesApp, events triggers, receipt of a written request form. The service end point address to which the invocation is directed should be included.
Technical	Invocation preconditions	Any pre-conditions that must be met by the consumer (authentication, additional input, etc.).
Technical	Business objects	Business objects transferred by the service.
Technical	Behavior characteristics	The criteria and conditions for successful interaction and any dependencies (on other service interactions, etc.). This should include any child services that will be invoked/spawned by this service (in addition to dependencies on other services).